Thompson sampling based Monte-Carlo planning in POMDPs

Monte-Carlo tree search (MCTS) has been drawinggreat interest in recent years for planning under uncertainty. One of the key challenges is the tradeoffbetween exploration and exploitation. To addressthis, we introduce a novel online planning algorithmfor large POMDPs using Thompson sampling basedMCTS that balances between cumulative and simple regrets.The proposed algorithm — Dirichlet-Dirichlet-NormalGamma based Partially Observable Monte-Carlo Planning (D2NG-POMCP) — treats the accumulatedreward of performing an action from a beliefstate in the MCTS search tree as a random variable followingan unknown distribution with hidden parameters.Bayesian method is used to model and infer theposterior distribution of these parameters by choosingthe conjugate prior in the form of a combination of twoDirichlet and one NormalGamma distributions. Thompsonsampling is exploited to guide the action selection inthe search tree. Experimental results confirmed that ouralgorithm outperforms the state-of-the-art approacheson several common benchmark problems

Efficient DCT-MCM Detection for Single and Multi-Antenna Wireless Systems

The discrete cosine transform (DCT) based multicarrier modulation (MCM) system is regarded as one of the promising transmission techniques for future wireless communications. By employing cosine basis as orthogonal functions for multiplexing each real-valued symbol with symbol period of T, it is able to maintain the subcarrier orthogonality while reducing frequency spacing to 1/(2T) Hz, which is only half of that compared to discrete Fourier transform (DFT) based multicarrier systems. In this paper, following one of the effective transmission models by which zeros are inserted as guard sequence and the DCT operation at the receiver is replaced by DFT of double length, we reformulate and evaluate three classic detection methods by appropriately processing the post-DFT signals both for single antenna and multiple-input multiple-output (MIMO) DCT-MCM systems. In all cases, we show that with our reformulated detection approaches, DCT-MCM schemes can outperform, in terms of error-rate, conventional OFDM-based systems

Performance analysis and optimization of DCT-based multicarrier system on frequency-selective fading channels

Regarded as one of the most promising transmission techniques for future wireless communications, the discrete cosine transform (DCT) based multicarrier modulation (MCM) system employs cosine basis as orthogonal functions for real-modulated symbols multiplexing, by which the minimum orthogonal frequency spacing can be reduced by half compared to discrete Fourier transform (DFT) based one. With a time-reversed pre-filter employed at the front of the receiver, interference-free one-tap equalization is achievable for the DCT-based systems. However, due to the correlated pre-filtering operation in time domain, the signal-to-noise ratio (SNR) is enhanced as a result at the output. This leads to reformulated detection criterion to compensate for such filtering effect, rendering minimum-mean-square-error (MMSE) and maximum likelihood (ML) detections applicable to the DCT-based multicarrier system. In this paper, following on the pre-filtering based DCT-MCM model that build in the literature work, we extend the overall system by considering both transceiver perfections and imperfections, where frequency offset, time offset and insufficient guard sequence are included. In the presence of those imperfection errors, the DCT-MCM systems are analysed in terms of desired signal power, inter-carrier interference (ICI) and inter-symbol interference (ISI). Thereafter, new detection algorithms based on zero forcing (ZF) iterative results are proposed to mitigate the imperfection effect. Numerical results show that the theoretical analysis match the simulation results, and the proposed iterative detection algorithms are able to improve the overall system performance significantly